Code framework for generic data extraction, analysis and reduction

ABSTRACT

A system and a method are provided for processing data from a computer that executes an application program. The system includes an extraction engine, a reduction program and an analysis program. The extraction engine retrieves execution data from the application program and records the data to nonvolatile memory as extract data. The reduction program reads the extract data from the memory and reduces the data into human readable format as reduce data. The analysis program analyzes the reduce data in accordance with operator-provided instructions and produces event data that identify at least one sequence associated with the execution data. A management console further displays the implemented processes. The extraction engine can further include a classes assigner for categorizing said execution data into classes. The reduction program can further include a structure builder for formatting said extract data. The reduction program can further include an event layout builder for formatting said event data.

CROSS REFERENCE TO RELATED APPLICATION

Pursuant to 35 U.S.C. §119, the benefit of priority from provisionalapplication 61/192,869, with a filing date of Sep. 17, 2008, is claimedfor this non-provisional application.

STATEMENT OF GOVERNMENT INTEREST

The invention described was made in the performance of official dutiesby one or more employees of the Department of the Navy, and thus, theinvention herein may be manufactured, used or licensed by or for theGovernment of the United States of America for governmental purposeswithout the payment of any royalties thereon or therefor.

BACKGROUND

The invention relates generally to data extraction techniques. Inparticular, the invention relates to data extraction reduction andanalysis, together with interfaces that operate to present theinformation from the data as events.

Modern software systems typically consist of a large number ofinterdependent software applications that run simultaneously on one ormore physical hardware systems. These applications employ well-definedsoftware interfaces in order to communicate data and invoke commands onremote software processes that enable the proper execution of thesoftware system as a whole. The success of large software systems isdirectly tied to the correct implementation of these softwareinterfaces.

SUMMARY

Conventional data extraction techniques yield disadvantages addressed byvarious exemplary embodiments of the present invention. In particular,these suffer from deficiencies related to portability to a variety ofplatforms.

Various exemplary embodiments provide a system and a method forprocessing data from a computer that executes an application program.The system includes an extraction engine, a reduction program and ananalysis program. The extraction engine retrieves execution data fromthe application program and records the data to nonvolatile memory asextract data. The reduction program reads the extract data from thememory and reduces the data into human readable format as reduce data.The analysis program analyzes the reduce data in accordance withoperator-provided instructions and produces event data that identify atleast one sequence associated with the execution data.

In various exemplary embodiments a management console displays theimplemented processes. The extraction engine can further include aclasses assigner for categorizing said execution data into classes. Thereduction program can further include a structure builder for formattingsaid extract data. The reduction program can further include an eventlayout builder for formatting said event data.

BRIEF DESCRIPTION OF THE DRAWINGS

These and various other features and aspects of various exemplaryembodiments will be readily understood with reference to the followingdetailed description taken in conjunction with the accompanyingdrawings, in which like or similar numbers are used throughout, and inwhich:

FIG. 1 is a block diagram view of a data extraction process;

FIG. 2 is a class diagram view for the structure of objects within thedata reduction component of the framework;

FIG. 3 is a process structure view of a sequence diagram for loadingevents;

FIG. 4 is a process structure view of a sequence diagram for loadingextracted data;

FIG. 5 is a process structure view of a sequence diagram for datareduction;

FIG. 6 is a class diagram view for the structure of objects within thedata extraction component of the framework;

FIG. 7 is a process structure view of a data extraction server;

FIG. 8 is a class diagram view the structure of objects within theextraction management console component of the framework;

FIG. 9 is a process structure view for a management console;

FIGS. 10-15 are tabular views of definition lists of parameters andoperators; and

FIGS. 16-23 are exemplary window views of a scenario that employs thedata reduction program.

DETAILED DESCRIPTION

In the following detailed description of exemplary embodiments of theinvention, reference is made to the accompanying drawings that form apart hereof, and in which is shown by way of illustration specificexemplary embodiments in which the invention may be practiced. Theseembodiments are described in sufficient detail to enable those skilledin the art to practice the invention. Other embodiments may be utilized,and logical, mechanical, and other changes may be made without departingfrom the spirit or scope of the present invention. The followingdetailed description is, therefore, not to be taken in a limiting sense,and the scope of the present invention is defined only by the appendedclaims.

The importance of the interfaces used in conventional display systemshas led software projects to typically perform data extraction, datareduction, and data analysis. These operations provide verification ofthe data and command instructions provided via software interfaces, andisolate problems should these occur. Traditionally, software projectshave developed support applications that perform the data extraction,data reduction, and data analysis functions required by the project.However, these support applications were focused on the data formats andinterfaces used by the particular software project and could not bereused on other projects without expending significant amounts time,money, and resources on code modification, rework, testing, andmaintenance.

Various exemplary embodiments provide a software framework that enablesdata extraction, data reduction and data analysis technologyapplications across a wide variety of software development projects. Theframework clearly and deliberately separates the data extraction, datareduction, and data analysis functions from the specific computerhardware, computer software, interface protocols, and data formats usedwithin a particular software development project. This facilitates thedevelopment of additional value-added data extraction, data reduction,and data analysis functions independent of the specific technologiesused by a particular software development project.

Additionally, the framework does not enforce the use of a particular setof hardware, software, interface protocols, or data formats on asoftware development project. The framework executes on a variety ofcomputer hardware running a variety of software operating systems, andallows the end- user (i.e., operator) to control the specifictechnologies used within their projects without adding additionaldependencies upon them. The framework thereby enhances the productivityof software projects by allowing the operator to gain the advantages ofdata extraction, data reduction, and data analysis technologies withoutincurring the expense of designing, developing, testing, and maintaininga custom set of tools that implement these functions.

Modern software systems typically consist of a large number ofinterdependent software applications running simultaneously on one ormore physical hardware systems. These applications employ well-definedsoftware interfaces in order to communicate data and invoke commands onremote software processes that enable the proper execution of thesoftware system as a whole. The success of large software systems isdirectly related to the correct implementation of these softwareinterfaces. Due to the importance of these interfaces, software projectstypically perform data extraction, data reduction, and data analysis toverify the data and command provided in software interfaces and toisolate problems if they occur.

Traditionally, a software project supports applications that perform thedata extraction, data reduction, and data analysis functions required bythe project for a specific platform. However, these support applicationshave been focused on the data formats an interlaces used by theparticular software project and could not be reused on other projectswithout expending significant amounts time, money, and resources on codemodification, rework, testing, and maintenance.

Due to these limitations, development of a set of tools was initiated toovercome these limitations by utilizing the capabilities of the C++computer programming language, freely available open source software,and object oriented design principles. Their concept involved thedevelopment of a framework that clearly separated the data extraction,data reduction, and data analysis functions from the software interlacesand data formats used by these functions. The actual implementation ofthe software interfaces and data formats would be left to the users ofthe framework.

The framework itself only contains functionality that is common acrossmultiple software projects, and enables the operator to customize thefeatures specific to their individual needs. Additionally, thisframework can be highly portable in order to execute on a variety ofcomputer hardware and software systems. By doing so, the framework couldbe usable by many different software projects thereby dramaticallyreducing the time, money, and resources required to implement dataextraction, data reduction, and data analysis capabilities in a softwaresystem.

A framework called Generic Data Extraction, Analysis and Reduction(GeDEAR) is currently being used by the Tomahawk Weapon Control Systemand the Littoral Combat Ship: Surface Warfare Mission Package: ModuleEngagement Controller software systems. The GeDEAR framework is beingconsidered by other programs as well, however at the time of thiswriting no other formal commitments have been received.

Software systems that utilize data extraction, data reduction, and dataanalysis capabilities typically organization these functions intoseveral software applications. The first is an application capable ofreceiving data from other software applications and recording the datato some form of persistent storage. In the GeDEAR framework, this isreferred to as the Extraction Server (ES). The second is an applicationthat reads the data from persistent storage, reduces the data into ahuman readable format, and analyzes the data according to rulesspecified by the operator. In the GeDEAR framework, this is referred toas the Data Reduction Program (DRP).

Some systems may include an additional application that controls theoperation of the ES. In the GeDEAR framework, this is referred to as theManagement Console (MC). The interactions between the software systemand the data extraction, data reduction, and data analysis capabilitiesare outlined in the following system example of using the GeDEARframework.

In this example, the software system consists of two physical hardwaremachines, computer-A and computer-B. Within these machines, severalsoftware applications are executing and communicating with each other.Additionally, one application is communicating with some externalinterface. This interface may be hardware or software. The ES runs on amachine and records any data provided thereto. The ES stores these datato some persistence (i.e., nonvolatile) storage. Within thisconfiguration, the state of the ES is monitored and controlled from theMC application. At some point in the future, the DRP opens thepersistent storage created by the ES, reads the stored events, appliessome logic to decode the events, and outputs the events to the operator.

FIG. 1 shows a block diagram 100 of the data extraction technique. Anexternal interface (I/F) communicates with computer-A 110 that executesan executable program application-A 115. The computer-A 110 communicateswith computer-B 120 that executes application-B 125 and application-C130. The computer-A 110 also communicates with an extraction server 140through an interface block 145. A management console 150 having aninterface 155 communicates with the server 140.

Application-B 125 communicates with application-A 115 and application-C130, and all three programs communicate with the server 140 through theinterface block 145. The server 140 provides data to memory storage 160through a format block 165. The data can be extracted from the storage160 to a reduction program 170 via a format block 175. The program 170can receive initial condition information from event descriptions 180 toprovide an event output 190.

The external interface 105, applications 115, 125 and 130 are specificto a project that uses the GeDEAR framework, which imposes no additionaldependency. The project using GeDEAR is at liberty to implement thesesections using any means necessary to complete the project. The server140, console 150 and program 170 represent core elements of the GeDEARframework and provide the functionality common across all data formatsand interface protocols.

The interface and format blocks 145, 155, 165, 175 are customizable bythe operator to meet their specific needs. Within the GeDEAR framework,these are referred to as plug-ins. Although the GeDEAR framework doesprovide some out-of-the-box reference implementations for thesecustomizable sections, the project using the framework may treat theseas optional and thereby may implement its own.

This separation between the interface/format blocks and the coreelements in FIG. 1 represents the result of the careful and deliberateisolation of the software interfaces and data formats used by dataextraction, data reduction, and data analysis capabilities from theparticular data being extracted. This separation provides the GeDEARframework with the flexibility required to enable generic dataextraction, data reduction, and data analysis capabilities. This enablesthe GeDEAR framework to focus on providing value-added data extraction,data reduction, and data analysis functions without delving into theintricacies of the underlying software interfaces and data formats.

The components of the GeDEAR framework operate with extracted datathrough a collection of abstract C++ classes that define the interfacethat all software interfaces and data formats conform to in order to besupported by the framework. These interfaces are further explainedsubsequently in this disclosure. From an end-user perspective, theseinterfaces take the form of plug-ins. These plug-ins can be written byinheritance from a base C++ class and implementing the required C++methods. The code for the plug-in can be compiled into a standalonelibrary that is loaded dynamically at run-time by the GeDEAR plug-inframework. All of the GeDEAR applications contain support for extensionand modification through the use of plug-ins.

The following sections of this document provide additional details ofthe three main components of the GeDEAR framework, the Data ReductionProgram (DRP), Extraction Server (ES), and Management Console (MC).Operators of the various framework components specify the plug-ins toload into the application as well as the software interfaces and dataformats that should be used for any given execution of the DRP, ES, orMC through the use of an extended markup language (XML) configurationfile. This configuration file is loaded during application startupprocessing. For example:

-   -   <drp>        -   <plugins>            -   <plugin                -   location=“/usr/lib”                -   name=“gedear_binary_format” />            -   <plugin                -   name=“gedear_output_mods” />        -   </plugins>        -   <eventdatasourceloader            -   name=“GeDear::Plugins::Formats::Binary”            -   extension=“DX*.dx” />        -   <structurebuilder            -   name=“GeDear::Plugins::Formats::Binary” />    -   </drp>

In this example, the operator instructs the DRP to load two plug-inlibraries. The first library is the named “gedear_binary_format” and islocated in /usr/lib. The second library is named “gedear_output_mods”and located in the current working directory.

The default EventDataSourceLoader can be set to“GeDear::Plugins::Formats::Binary” as the type. Additionally the DRP isinformed the file extension of this file type is in the format “Dx* .dx” This example also sets the default StructureBuilder to the type“GeDear::Plugins::Formats::Binary”.

There are no dependences between the components of the GeDEAR framework.The DRP may be used to reduce and analyze data that were not extractedfrom the ES, and the ES does not require use of the DRP for reductionand analysis. Likewise, the ES does not require the MC be used tomonitor and control it. The GeDEAR framework does not make use of anycustom written inter-application protocols.

FIG. 2 shows a class diagram 200 for the objects within the datareduction component of the framework. A set of DRP Human ComputerInterface (HCI) classes 210 consists of at least one interface 215 (formultiple loaders), which utilizes an implementation of the event datasource 220, and to at least one interface structure builder 230, whichutilizes on an interface event layout structure 235. The interface eventlayout structure consists of at least one interface event layout item250. The interface includes at least two implementations: Interfaceevent layout field 255 and event layout array 260. The interface eventlayout typed field 265 inherits from interface event layout field 255.The interface event layout array 260 may include one or more objects ofthe interface event layout item 250.

The object event layout 240 consists of one or more interface eventlayout structure 235 objects and one or more interface displayable 245objects. There are at least three implementations of interfacedisplayable 245. These are object constant 270, object field 275, andobject array 280. Object field 275 consists of a single interface eventlayout field 255 object. Object array 280 consists of one or moreinterface Displayable 245 objects.

In the flowchart 200, solid diamonds denote aggregation between items,open triangles denote inheritance and dash arrows indicate dependency.For example, the classes set 210 is aggregated with the interfaces 215and 230 to combine their functionalities at its penultimate block. Eachinterface provides block for aggregation represents a shell to implementthe instructions or establish formats therein. The classes providemethods, definitions and implementation procedures (i.e., executablefunctions) for reporting and analyzing data. The interface 250 inheritsdefinitions from the interface 255 and 260 to permit operation usinglower levels of instructions without controlling details of thosefunctions. The interface 220 provides an operator-defined declarationfor retrieving information fields within the data.

The Loading Events sequence begins with the operator informing the DRPHuman Computer Interface (HCI) to load events. The DRP HCI responds byinvoking the build method on the StructureBuilder instance to constructan EventLayout::Structure in accordance with the XML definitionprovided. The structure of the XML and the conversion of the XML into anEventLayout::Structure occurs via some implementation-specific means.

The DRP HCI then constructs an instance of EventLayout with the newlybuilt EventLayout::Structure. EventLayout iterates through the eventdisplay definitions contained with a separate XML structure andconstructs all the Displayable elements required by this event layout.EventLayout accomplishes this by invoking methods on theEventLayout::Structure in order to obtain the correct EventLayout::Itemneeded for display.

The Loading Data File sequence begins with the operator informing theDRP human computer interface to load a data file. The DRP HCI respondsby invoking the build method on each successive EventDataSourceLoaderobject until identifying a loader that can parse the provided data inthe file. The EventDataSourceLoader, through someimplementation-specific means, returns a new instance of EventDataSourceable to parse these data.

The processing events sequence begins with the DRP operator informingthe DRP HCI that processing should being. The DRP HCI then calls theEventDataSource for the MoreEvents method. If this source has moreevents, the DRP HCI retrieves the next event from the EventDataSource.From the event data, the DRP HCI locates the event definition able todisplay the extracted data. The DRP HCI then determines whether thisevent is valid for display, or else the event should be skipped as partof the filtering processing. This filtering is performed based uponuser-supplied criteria such as event number, event sequence number,event extraction time, or the actual values within fields within theevents.

For the condition in which the event is valid and should be displayed,the DRP HCI invokes the print method on the correct EventLayout object.The EventLayout object iterates through the Displayable objectscontained therein and invoke the print method on each respectiveDisplayable object. If this Displayable object actually represents aField, the underlying EventLayout::Field contained within theDisplayable object is called to return the actual value contained withinthe extracted data.

FIG. 3 shows an exemplary sequence diagram 300 for loading events intothe data reduction component of the framework. The sequence diagram 300details the general flow of execution within the DRP. A DRP Operatormark 310 initiates a first trigger 312 for loading events within a firstcycle 314 until completion. A DRP HCI Classes mark 320 initiates asecond trigger 322 for second and third cycles 324, 326. AStructureBuilder mark 330 initiates a third trigger 332 for a fifthcycle 334. An EventLayout mark 340 initiates a fourth trigger 342 havinga sub-trigger 344 for sixth and seventh cycles 346, 348.

The second cycle 324 builds structure until completion at the thirdtrigger 332, whereas the third trigger 326 creates classes untilcompletion at the fourth trigger 342. An EventLayout::Structure mark 350initiates a fifth trigger 352 with a construction loop 354 and a sixthtrigger 356. The fourth cycle 334 creates event layouts until completionat the fifth trigger 352. The fifth cycle 346 obtains names untilcompletion at the sixth trigger 356. A Display mark 360 provides aseventh trigger 362. The sixth cycle 348 creates layout structures untilcompletion at the seventh trigger 362.

FIG. 4 shows an exemplary sequence diagram 400 for loading extract datainto the DRP. A DRP Operator mark 410 initiates a first trigger 412 forloading data within a first cycle 414 until completion. A DRP HCIClasses mark 420 initiates the second trigger 422 for building theclasses within a second cycle 424 until completion. An Event Data SourceLoader mark 430 initiates a third trigger 432 that creates an Event DataSource 440.

FIG. 5 shows an exemplary sequence diagram 500 for the reduction ofevents within the DRP. A DRP Operator mark 510 initiates a first trigger512 for processing events within a first cycle 514 until completion. ADRP HCI Classes mark 520 initiates a second trigger 521 with asub-trigger 522 for building classes until completion. The first cycle514 processes events until completion at the second trigger 521. Thesub-trigger 522 initiates a third cycle 523 for ascertaining MoreEvents, a fourth cycle 524 for sequencing the Next Event, a find EventDefinition loop 525, a validation fifth cycle 526 for determining eventvalidity, and a sixth cycle 527 for printing events, all untilcompletion.

An Event Data Source mark 530 initiates third and fourth triggers 532,534 that respectively terminate the third and fourth cycles 523, 524. AnEvent Layout mark 540 initiates a fifth trigger 541 validates events ina seventh cycle 542, together with a sixth trigger 543 with asub-trigger 544 that prints in an eighth cycle 545.

A Display mark 550 initiates a seventh trigger 552 for obtaining displayvalues in a ninth cycle 554. An EventLayout::Structure mark 660initiates an eighth trigger 562; and an Event Layout::file 570 initiatesa ninth trigger 572. The seventh cycle 542 validates until completion atthe eighth trigger 562. The eighth cycle 545 prints until completion atthe seventh trigger 552, while the ninth cycle 554 obtains values untilcompletion at the ninth trigger 572.

FIG. 6 shows an exemplary class diagram 600 for identifying interfaceoperations and arguments. An Interface 610 operates in conjunction withan Extraction Engine 620, which interacts with an EventDataSink 630. TheInterface 610 includes operations for handling status, starting andstopping. The Engine 620 obtains Data, Status and establishes arepository. The Sink 630 provides closing, extraction, status andopening operations for receiving data for storage.

FIG. 7 shows an exemplary sequence model 700 for GeDEAR ExtractionServer Details. A GeDEAR Extraction Server 710 receives a user requestfrom an ExternalObject mark 720 through an external cycle 722. Withinthe Server 710, an Interface mark 730 initiates a first trigger 732 forhandling requests in a first (internal) cycle 734, and a second trigger736 for handling status in a second cycle 738.

An ExtractionEngine mark 740 initiates a third trigger 741 forsequencing requests in queuing loop 742, and a fourth trigger 743 forperforming requests in a third cycle 744 together with updating statusin an updating loop 745. An EventDataSink 750 initiates a fifth trigger752, which completes the third cycle 744 to perform requests. A separatethread 754 can also interact with the fourth trigger 743.

FIG. 8 shows an exemplary class diagram 800 for identifying HCI classes.A Console for HCI Classes 810 operates in conjunction with aServerFactory 820, which forwards information to a Server 830. TheFactory 820 builds the server, and obtains corresponding fields andnames. The Server 830 obtains status and submits requests forconnecting, disconnecting or obtaining exchange files.

FIG. 9 shows an exemplary sequence diagram 900 for Management Console(MC) Details. An MC Operator mark 910 initiates a first trigger 912 forselecting server type within a first cycle 914 until completion,enabling operator input in a parameter entry loop 916, and monitorbeginning in a second cycle 918. An MC HCI Classes mark 920 initiates athird trigger 921 for getting fields in third and fourth cycles 922, 923until completion, and initiates a fourth trigger 924 for validatinginput in a loop 925 and building a server in a fifth cycle 926 untilcompletion.

The first cycle 914 selects server types until completion at the secondtrigger 921, and the second cycle 918 begins monitoring until completionat the third trigger 924. A ServerFactory 930 initiates fourth, fifthand sixth triggers 932, 934 and 936, the last of which creating a serverin a sixth cycle 936 until completion. A Server 940 initiates a seventhtrigger that provides a terminus for the sixth cycle 936.

The parameters provided or created by these operations can besummarized, as provided by FIGS. 10-15, by a series of descriptions intabular form, with identification by name, description andimplementation. These lists, or portions of an overall list aredescribed briefly as follows.

FIG. 10 shows a first list 1000 that includes parameters with namesArray 1010, Constant 1020, Displayable 1030, DRP HCI Classes 1040 andEventDataSource 1050. For example, the first two parameters representobjects that are implemented by the GeDEAR framework. FIG. 11 provides asecond list 1100 that includes parameters with namesEventDataSourceLoader 1110, EventLayout 1120 and EventLayout::Array1130, all representing classes.

FIG. 12 shows a third list 1200 that includes parameters with namesEventLayout::Field 1210, EventLayout::ltem 1220 andEventLayout::Structure 1230. FIG. 13 indicates a fourth list 1300 thatincludes parameters with names EventLayout::TypedField 1310, Field 1320and StructureBuilder 1330.

FIG. 14 shows a fifth list 1400 that includes parameters with namesInterface 1410, ExtractionEngine 1420 and EventDataSink 1430. FIG. 15shows a sixth list 1500 that includes parameters with names Console HCIClasses 1510, ServerFactory 1520 and Server 1530.

Various exemplary embodiments describe the design that enablesdevelopment of GeDEAR tools, rather than particular GeDEAR applications.The following examples provide a typical usage scenario of the GeDEARData Reduction Program (DRP) to demonstrate the context of whenprocessing occurs and how the operator is informed of the processingstatus.

The first step involves loading the extracted data. In this sequence,the operator loads a file that contains extracted data. The first phaseis to inform the DRP that a data file should be loaded by selecting afile from the directory control or through the File menu. Upon selectingthe appropriate file, the DRP automatically opens the file and performsthe processing described in the sequence diagram 300.

FIG. 16 shows a first exemplary GeDEAR DRP window 1600. An upper menu1610 identifies four selections: input, events, filters and output. Anupper files contents box 1620 displays the directories and files in acascading list 1630 under the upper menu 1610, which selects “input” forthis example. An auxiliary contents box 1640 is disposed to the right ofthe box 1620 to provide file description, with an output range providedunderneath. A lower events header 1650 identifies six descriptors as atabular header. A lower events contents box 1660 provides an area fordisplay of summary information about the loaded file.

Upon completion of file loading, the operator may select a menu commandto generate an event summary. This summary lists the events within theloaded data file, when the events occur and how often the events occur.

FIG. 17 shows a second exemplary GeDEAR DRP window 1700. A highlightedfile 1710 can be selected within the box 1620. Events associated withthe file 1710 can be displayed in the box 1660 as a list 1720 withparameters corresponding to the descriptors in the header 1650.

The second step involves loading the events. In this sequence, theoperator loads events into the DRP to allow for their reduction anddisplay. The event file is an XML file that describes the data andoptionally defines the formatting for display in the box 1660. The firstphase is to inform the DRP that events should be loaded. This can beaccomplished through a file menu above the upper menu 1610 or through aload toolbar icon. FIG. 18 shows a third exemplary GeDEAR DRP window1800 with the file menu 1810, which can be expanded. This results in theDRP providing a standard file selection box to the operator enablingselection of the event file or files to load into the DRP.

The third step is performed internal to the DRP. This step involvesloading the event definitions described within the XML as prescribed inthe flowchart 300. FIG. 19 shows a fourth exemplary GeDEAR DRP window1900 in which events in the upper menu 1610 is selected. A window box1910 provides a list of event XML files, including a highlighted linethat identifies drp.dat.xml as the selected file.

Upon loading these events, the operator can be provided with a listingof all the events known to the DRP. FIG. 20 shows a fifth exemplaryGeDEAR DRP window 2000. The events are displayed as a list 2010 withinan expanded events content box 1660, with associated parameters listedunder their corresponding parameters in the header 1650.

The fourth step involves the operator activating at least one eventafter the events have been loaded. An active event represents the eventto be reduced from the extracted data and displayed to the operator. Anevent may be activated by either selecting the activate button orthrough right-click context menus. FIG. 21 shows a sixth exemplaryGeDEAR DRP window 2100. A selection 2110 of the events shown in the list2010 can be distinguished by highlighting. Activation opens a menu box2120 that includes an “activate” command, from which FIG. 22 displays aseventh exemplary GeDEAR DRP window 2200. The DRP denotes those eventsthat are active in the events table.

The fifth step involves reducing the data. With the extracted data fileselected and events loaded the operator may reduce the data. This causesthe DRP to display the active events within the data file in byperforming the processing described in sequence diagram 500. FIG. 23shows an eighth exemplary GeDEAR DRP window 2300. Selection of the“output” in upper menu 1610 yields within a window box 2310 a descriptorlist 2320 identifying actions performed from activation of the selectedfiles.

The GeDEAR framework has the following advantages over the old methods:

-   -   a. Well defined separation between the data formats and software        interfaces used within the system and the actual extraction,        reduction, and analysis capabilities;    -   b. Decoupling of the extraction and the reduction and analysis        capabilities;    -   c. Independent of the hardware used by the system;    -   d. Independent of the software used by the system; including        operating system, interface protocols, or programming language    -   e. Dynamic definition and application of the event descriptions        to the data reduction and analysis capabilities;    -   f. Run-time loading of data formats, software interfaces, and        output customizations used by the framework; and    -   g. Run-time modification of extraction, reduction, and analysis        functionality provided by the framework.

While certain features of the embodiments of the invention have beenillustrated as described herein, many modifications, substitutions,changes and equivalents will now occur to those skilled in the art. Itis, therefore, to be understood that the appended claims are intended tocover all such modifications and changes as fall within the true spiritof the embodiments.

1. An automated instruction execution processor to process data from acomputer that executes an application program, said processorcomprising: an extraction engine for retrieving execution data from theapplication program and recording said execution data to nonvolatilememory as extract data; a reduction program module for reading saidextract data from said memory and reducing said extract data into humanreadable format as reduce data; and an analysis program module foranalyzing said reduce data in accordance with operator-providedinstructions and producing event data that identify at least onesequence associated with said execution data, said sequence presenting aformat that includes first and second marks, said first mark initiatinga trigger to process an event within a cycle until completion at saidsecond mark.
 2. The processor according to claim 1, further comprising:a management console for displaying said retrieving, recording, reading,reducing, analyzing and producing processes.
 3. The processor accordingto claim 1, wherein said analysis program module includes an extendedmarkup language (XML) structure for building a template to display saidevent data.
 4. The processor according to claim 1, wherein said analysisprogram module further filters said sequence for acceptance withvalidity and to skip absent said validity.
 5. The processor according toclaim 1, wherein said extraction engine further includes a classesassigner for categorizing said execution data into classes.
 6. Theprocessor according to claim 5, wherein said reduction program modulefurther includes a structure builder for formatting said extract data.7. The processor according to claim 6, wherein said reduction programmodule further includes an event layout builder for formatting saidevent data.
 8. The processor according to claim 1, wherein said formatfurther includes a loop for processing said event data at said firstmark.
 9. A method for processing data from a computer that executes anapplication program, said method comprising: retrieving execution datafrom the application program via an extraction engine; recording saidexecution data to nonvolatile memory as extract data via said engine;reading said extract data from said memory via a reduction program;reducing said extract data into human readable format as reduce data viasaid reduction program; analyzing said reduce data via an analysisprogram, said analyzing being in accordance with operator-providedinstructions; producing event data that identify at least one sequenceassociated with said reduce data via said analysis program; anddisplaying said sequence in a format that includes first and secondmarks, said first mark initiating a trigger to process an event within acycle until completion at said second mark.
 10. The method according toclaim 9, further comprising: displaying said retrieving, recording,reading, reducing, analyzing and producing processes via a managementconsole.
 11. The method according to claim 9, wherein said producingoperation further includes building a template to display said eventdata via said analysis program, said template based on an extendedmarkup language (XML) structure.
 12. The method according to claim 9,wherein said analyzing operation further includes filtering saidsequence for acceptance via said analysis program, said filteringoperation to accept said sequence with validity and to skip saidsequence absent said validity.
 13. The method according to claim 9,wherein said extraction engine further includes categorizing saidexecution data into classes via a classes assigner.
 14. The methodaccording to claim 13, wherein said reduction program further includesformatting said extract data via a structure builder.
 15. The methodaccording to claim 14, wherein said reduction program further includesformatting said event data via an event layout builder.
 16. The methodaccording to claim 9, wherein said format further includes a loop forprocessing said event data at said first mark.